Notions of explainability and evaluation approaches for explainable artificial intelligence

Explainable Artificial Intelligence (XAI) has experienced a significant growth over the last few years. This is due to the widespread application of machine learning, particularly deep learning, that has led to the development of highly accurate models that lack explainability and interpretability. A plethora of methods to tackle this problem have been proposed, developed and tested, coupled with several studies attempting to define the concept of explainability and its evaluation. This systematic review contributes to the body of knowledge by clustering all the scientific studies via a hierarchical system that classifies theories and notions related to the concept of explainability and the evaluation approaches for XAI methods. The structure of this hierarchy builds on top of an exhaustive analysis of existing taxonomies and peer-reviewed scientific material. Findings suggest that scholars have identified numerous notions and requirements that an explanation should meet in order to be easily understandable by end-users and to provide actionable information that can inform decision making. They have also suggested various approaches to assess to what degree machine-generated explanations meet these demands. Overall, these approaches can be clustered into human-centred evaluations and evaluations with more objective metrics. However, despite the vast body of knowledge developed around the concept of explainability, there is not a general consensus among scholars on how an explanation should be defined, and how its validity and reliability assessed. Eventually, this review concludes by critically discussing these gaps and limitations, and it defines future research directions with explainability as the starting component of any artificial intelligent system

Classification of Explainable Artificial Intelligence Methods through Their Output Formats

Machine and deep learning have proven their utility to generate data-driven models with high accuracy and precision. However, their non-linear, complex structures are often difficult to interpret. Consequently, many scholars have developed a plethora of methods to explain their functioning and the logic of their inferences. This systematic review aimed to organise these methods into a hierarchical classification system that builds upon and extends existing taxonomies by adding a significant dimension—the output formats. The reviewed scientific papers were retrieved by conducting an initial search on Google Scholar with the keywords “explainable artificial intelligence”; “explainable machine learning”; and “interpretable machine learning”. A subsequent iterative search was carried out by checking the bibliography of these articles. The addition of the dimension of the explanation format makes the proposed classification system a practical tool for scholars, supporting them to select the most suitable type of explanation format for the problem at hand. Given the wide variety of challenges faced by researchers, the existing XAI methods provide several solutions to meet the requirements that differ considerably between the users, problems and application fields of artificial intelligence (AI). The task of identifying the most appropriate explanation can be daunting, thus the need for a classification system that helps with the selection of methods. This work concludes by critically identifying the limitations of the formats of explanations and by providing recommendations and possible future research directions on how to build a more generally applicable XAI method. Future work should be flexible enough to meet the many requirements posed by the widespread use of AI in several fields, and the new regulation

A Quantitative Evaluation of Global, Rule-Based Explanations of Post-Hoc, Model Agnostic Methods

Understanding the inferences of data-driven, machine-learned models can be seen as a process that discloses the relationships between their input and output. These relationships consist and can be represented as a set of inference rules. However, the models usually do not explicit these rules to their end-users who, subsequently, perceive them as black-boxes and might not trust their predictions. Therefore, scholars have proposed several methods for extracting rules from data-driven machine-learned models to explain their logic. However, limited work exists on the evaluation and comparison of these methods. This study proposes a novel comparative approach to evaluate and compare the rulesets produced by five model-agnostic, post-hoc rule extractors by employing eight quantitative metrics. Eventually, the Friedman test was employed to check whether a method consistently performed better than the others, in terms of the selected metrics, and could be considered superior. Findings demonstrate that these metrics do not provide sufficient evidence to identify superior methods over the others. However, when used together, these metrics form a tool, applicable to every rule-extraction method and machine-learned models, that is, suitable to highlight the strengths and weaknesses of the rule-extractors in various applications in an objective and straightforward manner, without any human interventions. Thus, they are capable of successfully modelling distinctively aspects of explainability, providing to researchers and practitioners vital insights on what a model has learned during its training process and how it makes its predictions

A comparative analysis of rule-based, model-agnostic methods for explainable artificial intelligence

The ultimate goal of Explainable Artificial Intelligence is to build models that possess both high accuracy and degree of explainability. Understanding the inferences of such models can be seen as a process that discloses the relationships between their input and output. These relationships can be represented as a set of inference rules which are usually not explicit within a model. Scholars have proposed several methods for extracting rules from data-driven machine-learned models. However, limited work exists on their comparison. This study proposes a novel comparative approach to evaluate and compare the rulesets produced by four post-hoc rule extractors by employing six quantitative metrics. Findings demonstrate that these metrics can actually help identify superior methods over the others thus are capable of successfully modelling distinctively aspects of explainability

A Policy-oriented Agent-based Model of Recruitment into Organized Crime

Criminal organizations exploit their presence on territories and local communities to recruit new workforce in order to carry out their criminal activities and business. The ability to attract individuals is crucial for maintaining power and control over the territories in which these groups are settled. This study proposes the formalization, development and analysis of an agent-based model (ABM) that simulates a neighborhood of Palermo (Sicily) with the aim to understand the pathways that lead individuals to recruitment into organized crime groups (OCGs). Using empirical data on social, economic and criminal conditions of the area under analysis, we use a multi-layer network approach to simulate this scenario. As the final goal, we test different policies to counter recruitment into OCGs. These scenarios are based on two different dimensions of prevention and intervention: (i) primary and secondary socialization and (ii) law enforcement targeting strategies.Comment: 15 pages, 2 figures. Paper accepted and in press for the Proceedings of the 2019 Social Simulation Conference (Mainz, Germany

Evolutionary advantages of turning points in human cooperative behaviour.

Cooperation is crucial to overcome some of the most pressing social challenges of our times, such as the spreading of infectious diseases, corruption and environmental conservation. Yet, how cooperation emerges and persists is still a puzzle for social scientists. Since human cooperation is individually costly, cooperative attitudes should have been eliminated by natural selection in favour of selfishness. Yet, cooperation is common in human societies, so there must be some features which make it evolutionarily advantageous. Using a cognitive inspired model of human cooperation, recent work Realpe-Gómez (2018) has reported signatures of criticality in human cooperative groups. Theoretical evidence suggests that being poised at a critical point provides evolutionary advantages to groups by enhancing responsiveness of these systems to external attacks. After showing that signatures of criticality can be detected in human cooperative groups composed by Moody Conditional Cooperators, in this work we show that being poised close to a turning point enhances the fitness and make individuals more resistant to invasions by free riders

Learning Dynamics and Norm Psychology Supports Human Cooperation in a Large-Scale Prisoner’s Dilemma on Networks

In this work, we explore the role of learning dynamics and social norms in human cooperation on networks. We study the model recently introduced in [Physical Review E, 97, 042321 (2018)] that integrates the well-studied Experience Weighted Attraction learning model with some features characterizing human norm psychology, namely the set of cognitive abilities humans have evolved to deal with social norms. We provide further evidence that this extended model—that we refer to as Experience Weighted Attraction with Norm Psychology—closely reproduces cooperative patterns of behavior observed in large-scale experiments with humans. In particular, we provide additional support for the finding that, when deciding to cooperate, humans balance between the choice that returns higher payoffs with the choice in agreement with social norms. In our experiment, agents play a prisoner’s dilemma game on various network structures: (i) a static lattice where agents have a fixed position; (ii) a regular random network where agents have a fixed position; and (iii) a dynamic lattice where agents are randomly re-positioned at each game iteration. Our results show that the network structure does not affect the dynamics of cooperation, which corroborates results of prior laboratory experiments. However, the network structure does seem to affect how individuals balance between their self-interested and normative choices

Exposure to selected preservatives in personal care products: case study comparison of exposure models and observational biomonitoring data

Exposure models provide critical information for risk assessment of personal care product ingredients, but there have been limited opportunities to compare exposure model predictions to observational exposure data. Urinary excretion data from a biomonitoring study in eight individuals were used to estimate minimum absorbed doses for triclosan and methyl-, ethyl-, and n-propyl- parabens (TCS, MP, EP, PP). Three screening exposure models (European Commission Scientific Commission on Consumer Safety [SCCS] algorithms, ConsExpo in deterministic mode, and RAIDAR-ICE) and two higher-tier probabilistic models (SHEDS-HT, and Creme Care & Cosmetics) were used to model participant exposures. Average urinary excretion rates of TCS, MP, EP, and PP for participants using products with those ingredients were 16.9, 3.32, 1.9, and 0.91 μg/kg-d, respectively. The SCCS default aggregate and RAIDAR-ICE screening models generally resulted in the highest predictions compared to other models. Approximately 60–90% of the model predictions for most of the models were within a factor of 10 of the observed exposures; ~30–40% of the predictions were within a factor of 3. Estimated exposures from urinary data tended to fall in the upper range of predictions from the probabilistic models. This analysis indicates that currently available exposure models provide estimates that are generally realistic. Uncertainties in preservative product concentrations and dermal absorption parameters as well as degree of metabolism following dermal absorption influence interpretation of the modeled vs. measured exposures. Use of multiple models may help characterize potential exposures more fully than reliance on a single model